SNOW-1843926: SPCS service events & metrics #1954

sfc-gh-ashen · 2024-12-13T22:02:58Z

Pre-review checklist

I've confirmed that instructions included in README.md are still correct after my changes in the codebase.
I've added or updated automated unit tests to verify correctness of my new code.
I've added or updated integration tests to verify correctness of my new code.
I've confirmed that my changes are working by executing CLI's commands manually on MacOS.
I've confirmed that my changes are working by executing CLI's commands manually on Windows.
I've confirmed that my changes are up-to-date with the target branch.
I've described my changes in the release notes.
I've described my changes in the section below.

Changes description

Design Document

1. Events Command: `snow spcs service events`

Purpose: Retrieve events for a specific service with advanced filtering options.

2. Metrics Command: `snow spcs service metrics`

Purpose: Fetch service metrics with similar filtering capabilities.

Change Motivation

Fill Monitoring Gaps: The current SnowCLI lacks support for retrieving service-specific metrics and events, limiting the ability to monitor and debug effectively.
Enhanced User Experience: Streamlined commands with filtering options for time, type, and scope will make monitoring easier and more efficient for users.

Alternate Approach: Standalone Command (`snow spcs events`)

Unified Command Consideration: Instead of separate commands for metrics and events, a unified snow spcs events command was considered. This would handle both service and compute pool telemetry.
- Pros:
  - Retrieve all telemetry types with a single command.
  - Expose more filtering capabilities, offering customers more flexibility.
- Cons:
  - Telemetry data for services is already supported under specific resource commands.
  - A generic command might add confusion, so a more intuitive design with separate commands is preferable.
Given these drawbacks, the current design with distinct commands was chosen for clarity and future extensibility.

Design Logic

Required Parameters

For both events and metrics, the following parameters are mandatory to ensure precise identification of the service and its resources:

name: Service identifier (Service name).
--container-name: Name of the container.
--instance-id: Instance ID of the service (starting with 0).

Filtering Options

Time-Based Filters:
- --since: Fetch telemetry data newer than the specified time (e.g., 1h, 5d).
- --until: Fetch telemetry data older than the specified time.
Pagination (for Events Only):
- --first <N>: Fetch only the first N events.
- --last <N>: Fetch only the last N events.
- --first and --last are mutually exclusive.
All Columns Display:
- --all: Fetch all columns from the event or metrics table.

Behavior for Metrics

No Filters Provided (--since, --until not used):
- Returns only the latest metrics for each unique metric type (grouped), ensuring deduplication of metrics.
With Any Filters (--since, --until):
- Returns all matching metrics from the event table based on the specified filters.

Data Formats

Events (Formatted Data)

TIMESTAMP	DATABASE NAME	SCHEMA NAME	SERVICE NAME	INSTANCE NAME	CONTAINER NAME	SEVERITY	EVENT NAME	EVENT VALUE
--	--	--	--	--	--	--	--	--

Metrics (Formatted Data)

TIMESTAMP	DATABASE NAME	SCHEMA NAME	SERVICE NAME	INSTANCE NAME	CONTAINER NAME	METRIC NAME	METRIC VALUE
--	--	--	--	--	--	--	--

Raw Data (`--all` Flag)

TIMESTAMP	START_TIMESTAMP	OBSERVED_TIMESTAMP	TRACE	RESOURCE	RESOURCE_ATTRIBUTES	SCOPE	SCOPE_ATTRIBUTES	RECORD_TYPE	RECORD	RECORD_ATTRIBUTES	VALUE	EXEMPLARS
--	--	--	--	--	--	--	--	--	--	--	--	--

Use Cases

`snow spcs service events`

Retrieve all events for a specific service:
```
snow spcs service events LOG_EVENT --container-name log-printer --instance-id 0
```
Output: Formatted table with key columns.

TIMESTAMP	DATABASE NAME	SCHEMA NAME	SERVICE NAME	INSTANCE NAME	CONTAINER NAME	SEVERITY	EVENT NAME	EVENT VALUE
2024-12-13 10:01:52.808692	TESTDB	PUBLIC	LOG_EVENT	0	log-printer	INFO	CONTAINER.STATUS_CHANGE	{ "message": "Running", "status": "READY" }
2024-12-14 22:27:25.420489	TESTDB	PUBLIC	LOG_EVENT	0	log-printer	INFO	CONTAINER.STATUS_CHANGE	{ "message": "Running", "status": "READY" }

2. **Retrieve a subset of events for a specific service:** - Limit the number of events: ```bash snow spcs service events LOG_EVENT --container-name log-printer --instance-id 0 --first 10 ``` Output: Formatted table showing the first 10 events.

Or fetch the last 10 events:

snow spcs service events LOG_EVENT --container-name log-printer --instance-id 0 --last 10

Output: Formatted table showing the last 10 events.

Narrow the time range using interval syntax:

Fetch events newer than the last 5 minutes:

snow spcs service events LOG_EVENT --container-name log-printer --instance-id 0 --since '5 minutes'

Output :

Fetch events older than 1 hour:

snow spcs service events LOG_EVENT --container-name log-printer --instance-id 0 --until '1 hour'

Output: Events within the specified time range.

Retrieve all events with all columns displayed:
```
snow spcs service events LOG_EVENT --container-name log-printer --instance-id 0 --all --last 1
```
Output: Raw table including all available columns.

Retrieve events formatted for JSON output:

snow spcs service events LOG_EVENT --container-name log-printer --instance-id 0 --last 1 --format json

Output (JSON):

[
    {
        "TIMESTAMP": "2024-12-14T22:27:25.420489",
        "DATABASE NAME": "TESTDB",
        "SCHEMA NAME": "PUBLIC",
        "SERVICE NAME": "LOG_EVENT",
        "INSTANCE NAME": "0",
        "CONTAINER NAME": "log-printer",
        "SEVERITY": "INFO",
        "EVENT NAME": "CONTAINER.STATUS_CHANGE",
        "EVENT VALUE": "{\n  \"message\": \"Running\",\n  \"status\": \"READY\"\n}"
    }
]

`snow spcs service metrics`

Retrieve metrics for a specific service:
```
snow spcs service metrics LOG_EVENT --container-name log-printer --instance-id 0
```
Output: Formatted table showing the latest metrics for each unique metric type.

TIMESTAMP	DATABASE NAME	SCHEMA NAME	SERVICE NAME	CONTAINER NAME	METRIC NAME	METRIC VALUE
2024-12-18 18:10:25.202000	TESTDB	PUBLIC	LOG_EVENT	log-printer	container.cpu.limit	1
2024-12-18 18:10:25.202000	TESTDB	PUBLIC	LOG_EVENT	log-printer	container.memory.requested	536870912
2024-12-18 18:10:25.202000	TESTDB	PUBLIC	LOG_EVENT	log-printer	container.memory.limit	6442450944
2024-12-18 18:10:25.202000	TESTDB	PUBLIC	LOG_EVENT	log-printer	container.cpu.requested	0.5
2024-12-18 18:10:08.957000	TESTDB	PUBLIC	LOG_EVENT	log-printer	container.cpu.usage	0.0004400012665396536
2024-12-18 18:10:08.957000	TESTDB	PUBLIC	LOG_EVENT	log-printer	container.memory.usage	1323008

Retrieve a subset of metrics with pagination:

Fetch the first 5 metrics:

snow spcs service metrics LOG_EVENT --container-name log-printer --instance-id 0 --first 5

Or fetch the last 5 metrics:

snow spcs service metrics LOG_EVENT --container-name log-printer --instance-id 0 --last 5

Output: Formatted table showing the first or last 5 metrics.

Retrieve metrics within a time range:

Fetch metrics newer than 1 hour:

snow spcs service metrics LOG_EVENT --container-name log-printer --instance-id 0 --since '1 hour'

Fetch metrics older than 2 hours:

snow spcs service metrics LOG_EVENT --container-name log-printer --instance-id 0 --until '2 hours'

Output: Metrics within the specified time range.

Retrieve metrics with all columns:

snow spcs service metrics LOG_EVENT --container-name log-printer --instance-id 0 --last 1 --all

Output: Raw table showing the last metric with all columns.

Retrieve metrics formatted for JSON output:

snow spcs service metrics LOG_EVENT --container-name log-printer --instance-id 0 --format json

Output (JSON):

[
    {
        "TIMESTAMP": "2024-12-14T22:27:25.420489",
        "SERVICE NAME": "LOG_EVENT",
        "INSTANCE NAME": "0",
        "CONTAINER NAME": "log-printer",
        "METRIC TYPE": "CPU_UTILIZATION",
        "VALUE": "75.4"
    }
]

src/snowflake/cli/_plugins/spcs/services/commands.py

sfc-gh-tkommineni · 2024-12-18T19:00:37Z

great job with design doc and description on the pr!

src/snowflake/cli/_plugins/spcs/common.py

src/snowflake/cli/_plugins/spcs/services/commands.py

src/snowflake/cli/_plugins/spcs/services/manager.py

src/snowflake/cli/_plugins/spcs/common.py

src/snowflake/cli/_plugins/spcs/services/commands.py

sfc-gh-pczajka · 2024-12-20T14:11:01Z

src/snowflake/cli/_plugins/spcs/common.py

+    elif isinstance(until, str) and until:
+        until_clause = f"and timestamp <= sysdate() - interval '{until}'"
+
+    return since_clause, until_clause


since_clause and until_clause are always used together - how about returning them as a single string (" AND ".join([since_clause, until_clause]))

Actually, we want customers have the flexibility of using these individually, Hence separated clauses

I meant it more as code reformat - functionality does not change. It felt weird for me that a function returns a tuple, which you always have to join into a string, rather than returning string ready to be used.

This is not blocking though, feel free to resolve

sfc-gh-pczajka · 2024-12-20T14:52:21Z

tests/spcs/test_services.py

+    "snowflake.cli.api.feature_flags.FeatureFlag.ENABLE_SPCS_SERVICE_METRICS.is_disabled"
+)
+@patch("snowflake.cli._plugins.spcs.services.manager.ServiceManager.execute_query")
+def test_latest_metrics(mock_execute_query, mock_is_disabled, runner):


Can we check what rows were printed as well (to check whether formatting / show all columns flag works as expected). snapshot fixture and pytest.mark.parametrize might be useful - you can check out test_help_messages for usage example :)

I don't see the changes - did you push the latest version?

I thought about adding

assert result.output == snapshot

using snapshot fixture - this way we could check the row formatting functions

You can do the same with expected_query (assert actual_query == snapshot) - this way the test code would become cleaner ;)

src/snowflake/cli/_plugins/spcs/services/commands.py

sfc-gh-pczajka

Some comments were marked as resolved without a change - did you forget to push?
(if not - please leave a comment why)

sfc-gh-ashen requested a review from a team as a code owner December 13, 2024 22:02

sfc-gh-ashen marked this pull request as draft December 13, 2024 22:03

sfc-gh-ashen force-pushed the SNOW-1843926-spcs-event-table-snow-cli-integration branch from 0700514 to 70cc699 Compare December 13, 2024 22:15

sfc-gh-ashen self-assigned this Dec 13, 2024

sfc-gh-tkommineni reviewed Dec 16, 2024

View reviewed changes

src/snowflake/cli/_plugins/spcs/services/commands.py Outdated Show resolved Hide resolved

sfc-gh-ashen changed the title ~~SNOW-1843926: SPCS service events&metrics~~ SNOW-1843926: SPCS service events & metrics Dec 18, 2024

sfc-gh-ashen marked this pull request as ready for review December 18, 2024 18:54

sfc-gh-ashen requested a review from a team as a code owner December 18, 2024 18:54

sfc-gh-tkommineni reviewed Dec 18, 2024

View reviewed changes

src/snowflake/cli/_plugins/spcs/common.py Outdated Show resolved Hide resolved

sfc-gh-tkommineni reviewed Dec 18, 2024

View reviewed changes

src/snowflake/cli/_plugins/spcs/common.py Show resolved Hide resolved

sfc-gh-pczajka reviewed Dec 20, 2024

View reviewed changes

src/snowflake/cli/_plugins/spcs/services/commands.py Outdated Show resolved Hide resolved

sfc-gh-pczajka reviewed Dec 20, 2024

View reviewed changes

src/snowflake/cli/_plugins/spcs/services/manager.py Outdated Show resolved Hide resolved

sfc-gh-pczajka reviewed Dec 20, 2024

View reviewed changes

sfc-gh-ashen commented Dec 20, 2024

View reviewed changes

src/snowflake/cli/_plugins/spcs/services/commands.py Show resolved Hide resolved

sfc-gh-tkommineni force-pushed the SNOW-1843926-spcs-event-table-snow-cli-integration branch from 72e4916 to d8e47b7 Compare January 1, 2025 22:43

sfc-gh-ashen added 8 commits January 6, 2025 11:28

spcs service events&metrics

570b198

snapshot and feature flag

fc67977

snapshot

3426e90

add new test

8971b78

fix test

5a7dc26

new table format and new test and new snapshot

0babe9f

fix comment

8f04e80

release notes

6789138

sfc-gh-tkommineni force-pushed the SNOW-1843926-spcs-event-table-snow-cli-integration branch 3 times, most recently from 5411ad2 to 5f38795 Compare January 7, 2025 15:46

sfc-gh-pczajka reviewed Jan 7, 2025

View reviewed changes

fix comments

281ca39

sfc-gh-tkommineni force-pushed the SNOW-1843926-spcs-event-table-snow-cli-integration branch from 5f38795 to 281ca39 Compare January 8, 2025 17:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SNOW-1843926: SPCS service events & metrics #1954

SNOW-1843926: SPCS service events & metrics #1954

sfc-gh-ashen commented Dec 13, 2024 •

edited

Loading

sfc-gh-tkommineni commented Dec 18, 2024

sfc-gh-pczajka Dec 20, 2024

sfc-gh-tkommineni Jan 8, 2025

sfc-gh-pczajka Jan 9, 2025

sfc-gh-pczajka Dec 20, 2024

sfc-gh-tkommineni Jan 2, 2025

sfc-gh-pczajka Jan 8, 2025

sfc-gh-pczajka left a comment

SNOW-1843926: SPCS service events & metrics #1954

Are you sure you want to change the base?

SNOW-1843926: SPCS service events & metrics #1954

Conversation

sfc-gh-ashen commented Dec 13, 2024 • edited Loading

Pre-review checklist

Changes description

1. Events Command: snow spcs service events

2. Metrics Command: snow spcs service metrics

Change Motivation

Alternate Approach: Standalone Command (snow spcs events)

Design Logic

Required Parameters

Filtering Options

Behavior for Metrics

Data Formats

Events (Formatted Data)

Metrics (Formatted Data)

Raw Data (--all Flag)

Use Cases

snow spcs service events

snow spcs service metrics

sfc-gh-tkommineni commented Dec 18, 2024

sfc-gh-pczajka Dec 20, 2024

Choose a reason for hiding this comment

sfc-gh-tkommineni Jan 8, 2025

Choose a reason for hiding this comment

sfc-gh-pczajka Jan 9, 2025

Choose a reason for hiding this comment

sfc-gh-pczajka Dec 20, 2024

Choose a reason for hiding this comment

sfc-gh-tkommineni Jan 2, 2025

Choose a reason for hiding this comment

sfc-gh-pczajka Jan 8, 2025

Choose a reason for hiding this comment

sfc-gh-pczajka left a comment

Choose a reason for hiding this comment

sfc-gh-ashen commented Dec 13, 2024 •

edited

Loading

1. Events Command: `snow spcs service events`

2. Metrics Command: `snow spcs service metrics`

Alternate Approach: Standalone Command (`snow spcs events`)

Raw Data (`--all` Flag)

`snow spcs service events`

`snow spcs service metrics`